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Abstract: The purposes of this study were to develop a culture specific critical thinking ability test 
for 6., 7. and 8. grade students in Turkey and to use it as an assessment instrument for giftedness. 
For these purposes, item pool involving 22 items was formed by writing items focusing on the 
current and common events presented in (Turkish) media from 2009 to 2011, general home 
problems and problems regarding to school, and by considering aspects of critical thinking 
dispositions which were stated by Facione (2000). The dispositions were considered as cognitive 
factors. The data was collected by applying the test to 162 ordinary and 65 gifted students who 
were enrolled in the middle and small scale elementary schools of East Anatolian Region of 
Turkey. The data on the instrument was analyzed for gathering reliability and validity evidence by 
using “Confirmatory Factor Analysis”, “Predictive Validity Analysis” and “Cronbach Alpha” 
reliability analysis. The results showed that reliability (a=.77) and validity evidence was enough to 
say that the instrument (Koksal’s Critical Thinking Ability Test) is appropriate to measure critical 
thinking ability of the elementary students. It was found the test was appropriate to discriminate 
gifted students from their ordinary counterparts. 
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1. Introduction 

The most frequently emphasized target of science education is to improve scientific literacy level of 
society (Project 2061, 2007). Definition of scientific literacy includes making informed decisions by 
using knowledge about science (Durant, 1993; Bybee, 1997; Gunn, Grigg, & Pomahac, 2007). In daily 
lives of people, situations in which individuals should make informed decisions for overcoming a 
problem are frequent and provide important examples for teaching scientific literacy in science 
education. In making informed decisions it is needed to develop higher-order thinking abilities 
including critical thinking. In parallel, critical thinking is also seen as a requirement to become 
scientifically literate (Vieira, Tenreiro-Vieira and Martins, 2011). Critical thinking ability is a means 
of selecting appropriate actions toward vague situations caused by existence of too much knowledge 
about the situations and actions of too many people who want to change our opinions in their way 
(Epstein, 1999). Lipman (1995) and Norris (1985) defined critical thinking as a process of critical 
examination of a claim by evaluating situated contexts of the claim and by focusing on its background 
assumptions, trustworthiness of evidence used to support the claim and structure of logical reasoning 
in reaching conclusions about the claim. While critical thinking is involved in making informed 
decisions, disposition to use critical thinking is also an effective factor for using critical thinking in 
making effective decisions. Norris (1992) indicated that insufficient disposition to use critical thinking 
might be result in failure to use critical thinking when it is required. Studies already represented a 
significant association between critical thinking dispositions and critical thinking skills (Colucciello, 
1997; Williams, Schmidt, Tilliss, Wilkins & Glasnapp, 2006; McGrath, 2003). Studies consistently 
show importance of critical thinking abilities and dispositions for learning. Barak, Ben Chaim & 
Zoller (2007) stated that critical thinking abilities are pre-requisites for learning and understanding 
science. Vieira, Tenreiro-Vieira and Martins (2011) also pointed out importance of critical thinking 
abilities by emphasizing use of critical thinking abilities in finding solutions to problems in biology 
and medicine, in making decisions regarding to scientific issues and in rejecting arbitrariness in the 
evaluation of argumentation related to scientific topics. The authors also established a theoretical 
association between scientific literacy and critical thinking and they suggested to need for increase 
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critical thinking abilities for developing scientific literacy. When looked at the studies on critical 
thinking dispositions, they also indicated that critical thinking dispositions are significantly and 
positively correlated with GPA (Facione, 2000; Kokdemir, 2003; Jenkins, 1998; Collins & 
Onwuegbuzie, 2000). Colucciello (1999) also showed that critical thinking dispositions are correlated 
with learning modes of students. The studies presented above have been showing the importance of 
critical thinking and the dispositions in science learning. 

Facione (2000) mentioned about six core critical thinking abilities including analysis, interpretation, 
inference, explanation, evaluation and self-regulation while Facione (2000), Facione and Facione 
(1992) described the seven components of critical thinking dispositions in their set of intellectual 
elements: open-mindedness, inquisitiveness, truth-seeking, analyticity, systematicity, self-confidence 
in reasoning and judicious. Open-mindedness is to take into account other individuals” opinions and 
to accept more valid opinion than individual’s own opinion. Analyticity is to apply reason and use 
evidence for understanding and explaining a problem while systematicity includes approaching a 
problem in a focused way. Inquisitive person has curiosity and is eager to learn about problems and to 
get knowledge about them. Truth-seeking involves desire to learn the best knowledge to understand 
nature of a problem. Confidence in reasoning is about making decisions by believing in ability to use 
reasoning effectively. Judicious individuals have ability to ask questions, to make judgment and to 
revise it (Colucciello, 1999; Facione, 2000). 

These abilities and dispositions are important for elementary school students in their future decisions 
which are important for their lives. Elementary level students are in a transition state from parent- 
dependent decision maker to independent decision maker. Independency brings higher-order problems 
regarding to daily life with itself. For example, buying more expensive things than previous parent- 
supported buying, gender-related social problems, selecting an appropriate high school for future 
carrier, making plan for future life and differentiating speculation from evidence-based claims are 
important samples requiring use of informed decision making by an independent person after their 
elementary school years. Elementary level education is a beginning point to design learning 
experiences in accordance with informed decision making processes on daily life problems. The most 
important component of selecting appropriate actions toward vague situations and in decision making 
is to have critical thinking ability (Epstein, 1999). In elementary level courses, critical thinking ability 
should be regarded as an effective component of all instructional activities focusing on decision 
making ability as a frequently emphasized aim of curriculums (Turkish Ministry of Education (MOE), 
2005; National Research Council [NRC], 1996; Malamitsa, Kokotas & Kasoutas, 2008). Critical 
thinking is a challenging component of teaching activities due to its higher-order nature. But, this is 
also an advantage for a group of students taking courses with ordinary students in spite of their higher- 
order cognitive abilities. In regular classrooms, gifted students are also involved and they have 
different cognitive characteristics and different educational requirements from their ordinary 
counterparts (Park & Oliver, 2009) Therefore, special education needs of students who have higher 
cognitive ability regarding to critical thinking than ordinary students should be determined to make 
instruction effective and appropriate for individual requirements. Gifted or high ability students have 
higher cognitive abilities than their ordinary counterparts (Rizza, McIntosh & McCunn, 2001; Kokis, 
Macpherson, Toplak, West & Stanovich, 2002). But determination of giftedness is generally based on 
cognitive abilities out of critical thinking in spite of clear importance of critical thinking in applying 
intelligence into daily life situations (McDaniel, 2005, Norris, 1985). Intellectual giftedness is closely 
associated with critical thinking (Linn & Shore, 2008). Actually there is a difference between gifted 
and ordinary students in terms of critical thinking. So measurement of critical thinking of students for 
gifted programs supports IQ-based diagnostic of them. Kettler (2014) compared gifted students with 
ordinary students by using two critical thinking tests and his findings showed that gifted students 
outperformed ordinary students on both of the tests. In Turkey selection of gifted students for 
nationally supported program of education is mostly based on IQ-test scores however critical thinking 
is a desired ability for gifted students in the program. Hence there is a need for developing critical 
thinking test for determining Turkish gifted students. 

For integration of critical thinking components into an enriched instruction, critical thinking ability of 
elementary level gifted students should be determined by using standardized instruments. In the 
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literature, there are some types of critical thinking ability intruments (Facione, Facione & Giancarlo, 
2001; Ennis &Weir, 1985; Watson & Galsser, 1980). But, none of the instruments focusing on critical 
thinking dispositions approach critical thinking dispositions as cognitive abilities which are not 
implicit and can be used for mere actions towards making informed decisions. In fact, critical thinking 
ability and critical thinking ability dispositions should be framed at the same theoretical structure since 
both of them are cognitive abilities (Facione, 2000). At the same time, the dispositions are abilities 
required in decision making situations. By considering critical thinking dispositions in decision 
making perspective, it is seen that they are ability components to reach a meaningful decision or 
conclusion. Some of the researchers’ findings supported the idea of considering the dispositions as 
cognitive critical thinking ability components in their studies by finding one-to-one correlation 
between critical thinking ability and critical thinking dispositions (Perkins, Jay & Tishman, 1993, 
McGrath, 2003). 

As another critic regarding to existent instruments (Keeley, Browne & Kreutzer,1982; Ennis & Weir, 
1985) of critical thinking ability, application and analysis of the data collected by the previous critical 
thinking tests in essay format are very hard for studying with high number of participants (Norris, 
1985). Therefore, there is some need to develop a valid and reliable critical thinking ability test. Based 
on this need, the purpose of this study is to develop a critical thinking ability test and to use it for 
discriminating gifted students from non-gifted (ordinary) ones?. 

2. Method 

The test developed in this study was called as Koksal’s Critical Thinking test. The items were written 
by the author by considering daily life problems emphasized in Turkish media from 2009 to 2011, 
general home problems and problems regarding to school. This way of taking into account three 
different contexts (media, home and school) was thought as more realistic to focus on daily life 
problems. And then, the items were evaluated by two teachers who were teaching science at the level 
of elementary education. One of the teachers was serving as a lecturer for gifted students at the center 
for gifted students. The teachers found the items appropriate in terms of readability, content, number 
of items, time assigned to complete instrument. Data for testing validity and reliability was collected 
by applying the instrument to students and taking the scores of them on science achievement and 
science performance from their teachers. Science achievement and performance scores were given by 
science teachers in students’ schools. Science achievement scores were collected by applying science 
content tests while performance scores were given by observing in-class studies and examining 
products the students produced for completing their science class tasks. These two measures were 
taken for getting predictive validity of the instrument. The application of the test took 20-25 min. After 
the collection of data, reliability analysis was conducted by running Cronbach alpha test in SPSS while 
confirmatory factor analysis was conducted by using AMOS program. In addition to these analyses, 
two groups including gifted and ordinary students were compared by using variance analysis. 

The structure of Koksal’s Critical Thinking test involved multiple-choice item format in which there 
were five choices. One of the choices was accepted as correct. Correctness of the choice was evaluated 
by two adult experts who were aware of the related factors of critical thinking. Although one choice 
was correct, there was also another choice including the state “I cannot give an answer to this 
question”. This choice was also accepted as proper answer due to its nature requiring making rational 
decision in time you were not able to give reaction when you did not feel yourself enough. Therefore, 
2 points were given for correct choice, while 1 point was given for the choice of “I cannot give an 
answer to this question”. And also zero point for an absent question or false answer was given to the 
participant. The range of the scores on one item included 0-2. One example of questions for 
systematicity factor is as the following. 

Question: You want to buy a printer for your computer. Your puipose of buying the printer is to print 
your writings in different colors in addition to black and white, to copy some written materials and to 
scan your documents. Which way do you prefer to buy such a printer? 

A. I compare the models which have the features I am interested in, in a table and I buy the most 
appropriate one. (correct choice) 

B. I buy the newest model of a known trademark in a known shopping center by directly going to 
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there. 

C. I buy the printer by asking about model, trademark and shopping place to my acquaintance 
who knows about computers and printers. 

D. I go to a shopping center and I directly buy the printer I like. 

E. I cannot give an answer to this question 

3. Participants 

The participants of this study included two different groups of the students; gifted (n=65) and ordinary 
(162). Seventy three of the ordinary participants were female while 84 of them were male 
(Missing=5). In the gifted group, 36 of the participants were female while 27 of them were male 
(Missing=2). The age of the participants ranged from 12 to 14. The ordinary students were enrolled in 
6., 7. and 8. grades of three different elementary schools in two middle-scale cities of Turkey, the 
gifted students took an education in 6., 7. and 8. grades of different elementary schools of a middle- 
scale city of Turkey. Furthermore, the gifted students who had IQ scores over 130 (WISC-R) were in 
the program of gifted education conducted in science and art centers. The frequencies regarding to 
grades are presented in table 1. 


Table 1. Frequencies of students in different grades 


Group 

Grade 

f 

Ordinary Students 

Sixth 

37 

Seventh 

35 

Eighth 

84 

Missing 

6 

Total 

162 

Gifted Students 

Sixth 

22 

Seventh 

28 

Eighth 

14 

Missing 

1 

Total 

65 


4 . Findings 

The findings of this study will be presented under the four sub-titles organized as confirmatory factor 
analysis, reliability analysis, predictive validity analysis, evidence on group difference. 

5. Confirmatory Factor Analysis Findings 

The instrument (Koksal’s Critical Thinking Ability Test) is based on the seven different disposition 
frame of Facione (2000). These dispositions to use critical thinking ability were used in this study due 
to fact that critical thinking dispositions and ability cannot be separated from each other and sufficient 
performance to think critically requires having dispositions to think critically. Moreover, the need to 
see both dispositions and performance together in a decision making situation requiring critical 
thinking drove us to evaluate critical thinking ability by framing the ability in disposition point of 
view. Keeping this idea in mind, ability test combining ability and dispositions at the same frame was 
structured. The factors of the test are presented in Table 2. 


Table 2. The factors of Koksal’s Critical Thinking Test and its items 


Factors 

Items 

Corresponding Abbreviations 

Factor-1 -Truth-seeking 

1,2,3,5 

GAE 

Factor-2-Judicious (Ability of Asking Questions) 

4,6,8 

SSB 

Factor-3-Analyticity 

10,12,14 

AN 

Factor-4-Systematicity 

13,15,17 

SIS 

Factor-5-Self-confidence in reasoning 

16,19,21 

KG 

Factor-6-Inquisitiveness 

18,20,22 

M 

Factor-7 -Open-mindedenes s 

7,9,11 

AF 

Total 

22 
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Looking on the frame of Facione (2000), dispositions based critical thinking ability model for 
confirmatory factor anlysis was established. The model has seven factors including open-mindedness, 
inquisitiveness, truth-seeking, analyticity, systematicity, self-confidence in reasoning and judicious. 
Figure 1 shows factor loads and regression values in the standardized model. As seen in the model 
regression weights regarding to AF (open-mindedness) are higher than 1. And also reliability value for 
this factor was found as .05, this indicates unacceptable reliability. Based on these two different 
evidence, AF factor was excluded from the model. 



Figure 1. Standardized regression values regarding to latent variables and obsen’ed variables in the first 

measurement 

After the exclusion of AF factor from the model, the model including remained factors were tested 
again for collecting validity evidence for the ability test. The modified model and results of the second 
analysis are presented in Figure 2. 
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Figure 2. Standardized regression values regarding to latent variables and obserx’ed variables after AF factor 

was discarded 


As seen in the Figure 2 showing results on standardized model including six factors, regression 
weights are high enough and not higher than 1 as an acceptable highest value. At the same time, 
factor loadings of the items are in acceptable ranges (higher than .30) (Costello & Osborne, 2005). 
Only one item had less factor loading value, but decreasing the number of items to two was not 
recommended by Costello and Osborne (2005). They suggest at least three items for one factor. So, 
item 12 was remained in the model. The fit indexes gathered from the analysis also supported the 
modified model. Fit indexes are presented in Table 3. 
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Table 3. Calculated fix indexes on the test scores and acceptable ranges regarding to the indexes 


Indexes 

Index Values 

Acceptable Ranges 

AGFI 

0.90 

0,85<AGFI<1,00 

CFI 

0.95 

0,90<CFI<1,00 

GFI 

0.93 

0,85<GFI<1,00 

RMSEA 

0.03 

0,00<RMSEA<0,08 


The indexes of CFI, AGFI and GFI for the scale scores of the students were higher than cut¬ 
off lower limits of .90 and .85 for CFI, AGFI and GFI, respectively (Hoyle, 2000; Marsh, 
Balia and McDonald, 1988; Schermelleh-Engel and Moosbrugger, 2003). RMSEA was also 
smaller than .08 as an acceptable value for each analysis (Raykov and Marcoulides, 2006). 
These values indicate good fit for the data collected by the instrument. After the examination 
for fit indexes, relationship among factors and total scores were investigated and it was found 
that all of the factors were significantly related to each other. Also they are significantly 
associated with total score. Correlational evidence (see table 4) is another support for the 
factor structure of the instrument. 


Table 4. Pearson-product moment correlations among the factors of the critical thinking test 


Factors 

Factors 

1 

2 

3 

4 

5 

6 

Factor 1 

1 

.34 

.29 

.17 

.35 

.24 

Factor 2 


1 

.27 

.22 

.31 

.28 

Factor 3 



1 

.18 

.31 

.30 

Factor 4 




1 

.30 

.38 

Factor 5 





1 

.37 

Factor 6 






1 

Total 

.66 

.64 

.59 

.59 

.68 

.66 


Note: All of the correlations are significant at the level of .05. 


6. Reliability Findings 

The factor structure of the instrument was tested in “confirmatory factor analysis”, but there was a 
need to know about reliability of the scores on the instrument. The reliability value of the scores on the 
whole instrument are higher than .75. But, the reliability values regarding to each factor are not as 
good as whole-test reliability. This is an expected situation, since the more number of the items 
regarding to a factor decreases, the more reliability value regarding to this factor decreases. In fact, 
reliabilities of five factors except for factor 3 are in acceptable ranges (see table 5) (Buyiikoztiirk, 
2002). This low level reliability might have been caused by low factor loading of item 12. But this 
factor is associated with all other factors and total score on the instrument. Therefore, this situation 
was ignored for holding three items for factor 3. 


Table 5. Interned reliability values of the scores on the factors of critical thinking test 


Factors 

Numbers of Items 

Cronbach Alpha Coefficient 

Factor 1 

4 

.50 

Factor 2 

3 

.56 

Factor 3 

3 

.24 

Factor 4 

3 

.58 

Factor 5 

3 

.47 

Factor 6 

3 

.63 

Total Test 

19 

.77 


7. Predictive Validity Findings 

The instrument was validated on factor structure and its reliability was found enough, but further 
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evidence for predictive validity was needed to reach more advisable findings. To collect evidence for 
predictive validity, science achievement and science performance scores of 81 ordinary students were 
taken after the ability test was applied. The results of the correlational analyses showed that total 
critical thinking scores of the students are significantly related to science achievement and 
performance scores. The findings are presented in table 6. 


Table 6. Correlation values of critical thinking ability scores of the participants with their science course 

achievement scores and performance scores 


Correlation Pairs (n=81) 

Pearson-Product Moment Correlation 

Total Critical Thinking Scores- Science 
Course Achievement Scores 

.35 

Total Critical Thinking Scores- Science 
Course Performance Scores 

.25 


Note: All of the correlations are significant at the level of .05. 


7. Findings on Comparison of Gifted and Ordinary Students in Terms of Critical 
Thinking Ability 

To get further evidence for the validity of the instrument and to collect evidence for its discrimination 
function between gifted and ordinary students, two different groups who had different cognitive ability 
characteristics were compared. Literature showed that gifted students have higher cognitive abilities 
than their ordinary counterparts (Rizza, McIntosh & McCunn, 2001; Kokis, Macpherson, Toplak, 
West & Stanovich, 2002). In this study, critical thinking is accepted as a type of cognitive ability, so 
the difference between gifted and ordinary students in terms of critical thinking test is accepted as an 
evidence for norm-group validity of the test. At the same time, difference between the two groups is 
accepted as evidence for its discrimination function. The ANOVAs showed statistically significant 
differences between gifted and non-gifted students in terms of all factor scores and whole-test scores 
(see table 7). 

Table 7. Findings regarding to the difference between the scores of the gifted and ordinary students on Koksal’s 

Critical Thinking Ability test and its Sub-components. 


Factors 

Groups 

N 

Mean 

S 

F 

P 

GAE 

Ordinary 

162 

.87 

.55 

9.78 

.002* 

Gifted 

65 

1.14 

.67 

Total 

227 

.94 

.60 

SSB 

Ordinary 

162 

.67 

.63 

28.70 

.000* 

Gifted 

65 

1.19 

.71 

Total 

227 

.82 

.69 

ANA 

Ordinary 

162 

.68 

.55 

24.24 

.000* 

Gifted 

65 

1.09 

.57 

Total 

227 

.80 

.58 

sis 

Ordinary 

162 

.89 

.67 

22.04 

.000* 

Gifted 

65 

1.35 

.66 

Total 

227 

1.03 

.70 

KG 

Ordinary 

162 

.97 

.61 

35.93 

.000* 

Gifted 

65 

1.50 

.60 

Total 

227 

1.12 

.66 

M 

Ordinary 

162 

1.07 

.65 

53.55 

.000* 

Gifted 

65 

1.71 

.44 

Total 

227 

1.26 

.66 

Total 

Ordinary 

162 

.86 

.35 

77.41 

.000* 

Gifted 

65 

1.32 

.38 

Total 

227 

.99 

.41 


Note: All of the differences are significant at the level of .008 (Bonferroni Adjustment). 


The findings on the comparisons are an indication of the potential use of the instrument for 
determining gifted students at the level of elementary school. For using the instrument for 
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determination of gifted student, it is recommended to use scores between .96 and 1.24 as potentially 
gifted, scores between 1.25-1.69 as gifted and scores between 1.70-2.00 as highly gifted. These ranges 
are coming from matching the score ranges in 1 SD above/below the mean score of the ordinary 
students with the score ranges in 1SD above/below the mean score of the gifted students. But, this way 
of determination should be supported by using IQ tests and other diagnostic tests. 

7. Discussion and Conclusion 

The evidence on reliability and validity regarding to the instrument supported to validation of 
anticipated theoretical structure of the instrument and internal consistency of the items. But the factor 
regarding to “open-mindedness” (AF) was not found as reliable and valid for measuring this aspect of 
critical thinking ability. Kokdemir (2003) showed that some of the items measuring “open- 
mindedness” are strongly related to the factors regarding to other aspects of critical thinking. 
Therefore, the items of the “open-mindedness” factor can be related to different factors investigated in 
this study. This question should be studied by establishing theoretical association of the items with 
other factors of critical thinking. As another important point to discuss, the factor related to 
“analyticity” had low reliability and item 12 of this factor had not as high as expected factor loading 
on the “analyticity”. In spite of this problem, the researchers decided to hold this factor and item 12 in 
the model due to the fact that the factor is related to core ability of critical thinking and item 12 
important to keep the number of items high enough (three) for reaching expected valuation for making 
statistical analysis (Vieira, Tenreiro-Vieira & Martins, 2011; Costello & Osborne, 2005). The 
reliability problem might also be associated with number of participants taking the test, if the number 
of the participants is increased, more comprehensive analysis of reliability on the scores might be 
done. Cronbach alpha reliability is a sensitive technique to number of the participants (Henson, Kogan 
& Vacha-Haase, 2001). 

The other important issue on the instrument is about relationship between the scores coming from the 
critical thinking test and, science achievement and science performance assessments. These results 
supported predictive validity of the critical thinking ability, since Facione (2000), Kokdemir (2003) 
and Collins & Onwuegbuzie (2000) also showed existence a relationship between critical thinking 
ability scores and school achievement scores. Another side of this results is that science achievement 
and critical thinking ability as two components of scientific literacy are found as related factors as 
expected from the literature (Vieira, Tenreiro-Vieira & Martins, 2011; Gunn, Grigg & Pomahac, 
2007). Barak, Ben Chaim & Zoller (2007) took this situation further and said that critical thinking 
ability is pre-requisite for learning science. 

As another finding, comparison of the groups provided an important result supporting the function of 
the instrument in discriminating the gifted students from their ordinary counterparts. In this study, 
critical thinking ability was regarded as a cognitive variable, so it was expected that gifted students 
should have higher scores from the instrument application. According to Rizza, McIntosh and 
McCunn (2001), Kokis et al. (2002), gifted students have higher cognitive abilities than their ordinary 
counterparts. As expected, the results showed a statistically significant difference between the gifted 
and non-gifted students in favor of gifted students. This is also an indication of norm group validity of 
the instrument. 

In summary, the instrument has strong structure, predictive and “norm group comparison” validity and 
acceptable reliability. It can be recommended to use the instrument for determining students’ general 
level of critical thinking abilities. And also, the instrument can be used for selecting student for special 
programs including critical thinking purposes. As another field to use the instrument, gifted students 
requiring a special education can be determined by using this instrument as an additional measurement 
tool to intelligence tests and other instruments. 
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